Skip to content

Conversation

@troylar
Copy link

@troylar troylar commented Aug 10, 2017

Our requests were failing because the ElasticSearch library creates a Connection:Keep-Alive header by default. Amazon expects the request to be signed with the header Connection:close. Overriding this header allowed us to successfully query the AWS ES cluster.

This pull request simply allows you to switch between "Keep-Alive" and "close."

Copy link
Author

@troylar troylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the error we are getting from AWS:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'GET\n/_cluster/health\n\naccept:application/json\nconnection:close\ncontent-type:application/json\nhost:search-poc-test-vp7ggjs3lreczj7w7b3kzs5qqy.us-east-2.es.amazonaws.com\nx-amz-date:20170810T211424Z\n\naccept;connection;content-type;host;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae

protected override System.Net.HttpWebRequest CreateHttpWebRequest(RequestData requestData)
{
var request = base.CreateHttpWebRequest(requestData);
request.Headers.Remove("Connection");
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is duplicate code, but the request object is different for both methods.

@bcuff
Copy link
Owner

bcuff commented Aug 10, 2017

Any chance you're going through a proxy that might be messing with the HTTP? We are successfully connecting directly to an AWS Elasticsearch service cluster using https:// and have never encountered this. Also using Keep Alive is important for performance reasons.

@troylar
Copy link
Author

troylar commented Aug 10, 2017 via email

@bcuff
Copy link
Owner

bcuff commented Aug 10, 2017

You can try omitting the connection header from the signature. I don't think it needs to be part of the signature. See f3f3a52

If that fixes the issue let me know and we can include that change. I would hate to turn off keep-alive by default.

@troylar
Copy link
Author

troylar commented Aug 10, 2017 via email

@troylar
Copy link
Author

troylar commented Aug 11, 2017

So removing the Connection header worked as well.

@bcuff
Copy link
Owner

bcuff commented Aug 11, 2017

Cool. I'll merge that commit and publish tomorrow morning. Going to close this PR. thanks for reporting the issue.

@bcuff bcuff closed this Aug 11, 2017
@troylar
Copy link
Author

troylar commented Aug 11, 2017

@bcuff FYI -- this where the keep-alive is being set in the Elasticsearch-net code . . .It only happens for the DOTNETCORE directive. (Line 225)

https://github.com/elastic/elasticsearch-net/blob/70571f15a8fcd5b6397b9aa9c3a9c81d37014b51/src/Elasticsearch.Net/Connection/HttpConnection-CoreFx.cs

@troylar
Copy link
Author

troylar commented Aug 11, 2017

And where the addition was made two months ago. Just wanted to give a full context. Thanks for making this change.

elastic/elasticsearch-net@0e66e5b

@bcuff
Copy link
Owner

bcuff commented Aug 11, 2017

Ah, that makes sense. New release is up https://www.nuget.org/packages/Elasticsearch.Net.Aws/2.3.5
@troylar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants